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On  the  Probability  of  Error  and  Stochastic  Resonance  in  Discrete  Memoryless 

Channels 

Daniel  T  Gebremicheal 

In  this  thesis,  we  studied  the  performanee  of  Diserete  Memoryless  Channels  (DMC), 
arising  in  the  eontext  of  cooperative  underwater  wireless  sensor  networks.  We  formulated  an 
analytic  relationship  that  relates  the  average  probability  of  error  to  the  systems  parameters,  the 
signal  amplitude,  the  decision  threshold  and  the  noise  power  values.  First  we  studied  the  trade¬ 
off  between  the  signal  amplitude  and  the  decision  threshold  in  the  special  case  (2,  2)  DMC. 
Following  our  (2,  2)  DMC  model  we  proposed  a  symmetric  decision  threshold  to  formulate  our 
analytical  relationship  for  the  average  probability  of  error  based  on  four  arbitrarily  defined 
regions  of  interest.  In  order  to  design  a  resilient  system,  conditional  probabilities  of  error  are 
defined  with  respect  to  the  optimum  zero  threshold  in  the  (2,  3)  DMC. 

We  analyzed  the  stochastic  resonance  (SR)  phenomenon  impact  upon  the  performance 
limits  of  a  distributed  underwater  wireless  sensor  networks  operating  with  limited  transmitted 
power  and  computational  capabilities.  We  focused  on  the  threshold  communication  systems 
where,  due  to  the  underwater  environment,  non-coherent  communication  techniques  are  affected 
both  by  noise  and  threshold  level.  The  binary-input  ternary-output  channel  is  used  as  a 
theoretical  model  for  the  DMC. 

To  further  deduce  the  relationship  between  the  signal  amplitude  and  the  decision 
threshold,  we  classified  the  average  probability  of  error  into  its  respective  probabilities  of 
interest;  the  probability  of  false  alarm,  probability  of  correct  detection,  probability  of  miss  and 
probability  of  correct  rejection.  We  studied  how  the  probability  of  correct  detection  and  false 
alarm  varies  with  our  symmetric  decision  threshold  theta  and  the  noise  variance  sigma.  In 
addition  to  that  we  investigated  how  the  probability  of  correct  rejection  and  miss  varies  with  our 
system  parameters. 

We  also  investigated  how  a  sub  threshold/super  threshold  signal  determines  the  optimum 
system  performance  in  the  case  of  the  (2,  2)  and  the  (2,  3)  DMCs.  We  found  the  optimum  value 
of  the  additive  noise  power  that  results  in  a  better  system  performance  in  bistable  systems  as 
function  of  detector  threshold  and  the  signal  amplitude. 

Our  findings  show  how  the  combination  of  the  (2,  2)  DMC  and  the  simple  symmetric  (2, 
3)  DMC  yields  a  better  signal  detection  in  the  presence  of  additive  Gaussian  noise.  In  the  case  of 
our  proposed  (2,  3)  DMC  for  the  underwater  distributed  wireless  sensor  networks,  the  probability 
associated  with  zero  optimum  crossing  threshold  remains  constant.  Thus,  in  order  for  the 
probability  of  detection  to  be  maximized  by  minimizing  the  probability  of  false  alarm  and  miss, 
the  detection  scheme  needs  to  incorporate  suprathreshold  signal  and  symmetric  threshold  around 
the  zero  optimum  threshold  in  the  case  of  (2,  2)  DMC. 
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Summary 


In  this  thesis,  we  studied  the  performanee  of  Diserete  Memoryless  Channels  (DMCs), 
arising  in  the  eontext  of  eooperative  underwater  wireless  sensor  networks.  We  formulated  an 
analytie  relationship  that  relates  the  average  probability  of  error  to  the  systems  parameters,  the 
signal  amplitude,  the  deeision  threshold  and  the  noise  power  values.  We  first  studied  the  trade¬ 
off  between  the  signal  amplitude  and  the  deeision  threshold  in  the  speeial  ease  (2,  2)  DMC. 
Following  our  (2,  2)  DMC  model  we  proposed  a  symmetrie  deeision  threshold  in  order  to  find 
the  average  probability  of  error  based  on  four  arbitrarily  defined  regions  of  interest.  In  order  to 
design  a  resilient  system  eondition  probabilities  of  error  are  defined  with  respeet  to  the  optimum 
zero  threshold  in  the  (2,  3)  DMC. 

We  analyzed  the  stoehastie  resonanee  (SR)  phenomenon  impaet  upon  the  performanee 
limits  of  a  distributed  underwater  wireless  sensor  networks  operating  with  limited  transmitted 
power  and  eomputational  eapabilities.  We  foeused  on  the  threshold  eommunieation  systems 
where,  due  to  the  underwater  environment,  non-eoherent  eommunieation  teehniques  are  affeeted 
both  by  noise  and  threshold  level.  The  binary-input  ternary-output  ehannel  is  used  as  a 
theoretieal  model  for  the  DMC. 

To  further  deduee  the  relationship  between  the  signal  amplitude  and  the  deeision 
threshold,  we  elassified  the  average  probability  of  error  into  its  respeetive  probabilities  of 
interest;  the  probability  of  false  alarm,  probability  of  eorreet  deteetion,  probability  of  miss  and 
probability  of  eorreet  rejeetion.  We  studied  how  the  probability  of  eorreet  deteetion  and  false 
alarm  varies  with  our  symmetrie  deeision  threshold  theta  and  the  noise  varianee  sigma.  In 
addition  to  that  we  investigated  how  the  probability  of  eorreet  rejeetion  and  miss  varies  with  our 
system  parameters. 

We  also  investigated  how  a  sub  threshold/super  threshold  signal  determines  the  optimum 
system  performanee  in  the  ease  of  the  (2,  2)  and  the  (2,  3)  DMCs.  We  found  the  optimum  value 
of  the  additive  noise  power  that  results  in  a  better  system  performanee  in  bistable  systems  as 
funetion  of  deteetor  threshold  and  the  signal  amplitude. 

Our  findings  show  how  the  eombination  of  the  (2,  2)  DMC  and  the  simple  symmetrie  (2, 
3)  DMC  yields  a  better  signal  deteetion  in  the  presenee  of  additive  Gaussian  noise.  In  the  ease  of 
our  proposed  (2,  3)  DMC  for  the  underwater  distributed  wireless  sensor  networks,  the  probability 
assoeiated  with  zero  optimum  erossing  threshold  remains  eonstant.  Thus,  in  order  for  the 
probability  of  deteetion  to  be  maximized  by  minimizing  the  probability  of  false  alarm  and  miss, 
the  deteetion  seheme  needs  to  ineorporate  suprathreshold  signal  and  symmetrie  threshold  around 
the  zero  optimum  threshold  in  the  ease  of  (2,  2)  DMC. 


Chapter  1 
Introduction 

In  this  thesis,  we  investigated  probability  of  error  and  Stoehastie  Resonanee  (SR)  effeet  for 
the  binary-input  ternary-output  (2,  3)  Diserete  Memoryless  Channel  (DMC).  We  revisited  the 
BSC  whieh  is  a  speeial  ease  of  the  (2,  3)  DMC.  We  studied  how  the  probability  of  error  varies 
for  fixed  value  of  the  threshold  as  the  signal  to  noise  ratio  inereases.  We  investigated  the  trade¬ 
off  between  the  signal  amplitude  and  the  deeision  threshold  value  for  a  different  value  of  noise 
power. 

In  order  to  study  SR,  we  modeled  the  (2,  3)  DMC  by  a  physieal  eommunieation  ehannel.  We 
proposed  a  symmetrie  deeision  threshold  and  eonditional  probabilities  in  order  to  maximize 
target  deteetion  and  traeking  in  the  ease  of  underwater  distributed  wireless  sensor  networks.  In 
the  presenee  of  a  Gaussian  noise,  the  SR  phenomenon  is  observed  over  a  eertain  range  of 
threshold  values.  We  foeused  on  the  additive  Gaussian  noise  in  whieh  ease  we  determined  the 
interval  under  whieh  our  system  exhibit  SR.  The  main  eontributions  of  this  thesis  eonsist  in 
deriving  the  average  probability  of  error  in  the  (2,  3)  DMC  and  determining  the  interval  by 
whieh  the  SR  effeet  is  observed. 

1.1  Thesis  Structure 

The  rest  of  the  thesis  is  struetured  as  follows 

Chapter  2  provides  the  baekground  eoneepts  and  the  literature  reviews,  whieh  will  be  used 
throughout  the  thesis.  We  also  review  the  (2,  2)  DMC. 

Chapter  3  reviews  and  derives  the  probability  of  error  in  the  (2,  3)  DMC.  We  investigate  the 
tradeoff  between  the  signal  amplitude  and  the  deeision  threshold  for  optimum  system 
performanee.  We  deduee  the  relationship  between  the  signal  amplitude  and  the  deeision 
threshold  theta.  Moreover,  we  investigate  how  the  probabilities  of  false  alarm  and  deteetion  vary 
with  respeet  to  the  symmetrie  deeision  threshold  theta.  Similarly  we  study  how  the  probability  of 
miss  and  eorreet  rejeetion  are  also  affeeted  by  the  seleetion  of  the  threshold  0  and  noise  power  a. 

Chapter  4  analyzes  the  effeets  of  SR  and  the  physieal  model  of  the  (2,  3)  DMC  in  presenee  of 
Gaussian  noise.  We  eompare  the  performanee  of  the  (2,  2)  and  (2,  3)  DMCs  in  the  eontext  of 
SR.  Also  we  derive  the  optimum  value  of  the  noise  that  results  in  SR  in  the  ease  of  the  (2,  2) 
DMC  and  the  (2,  3)  DMC.  We  also  eheek  the  fundamental  theorem  in  the  ease  of  SR,  the 
forbidden  interval  theorem  (FIT). 
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Chapter  5  engages  a  diseussion  about  the  results  and  future  work  which  can  enhance  the  signal 
detection  and  application  of  our  proposed  (2,  3)  DMC  in  the  case  of  underwater  distributed 
wireless  sensor  networks. 


1.2  Thesis  Significance  and  Contribution 

This  research  contributes  to  Dr.  Cotae’s  research  titled  “Information  -  Driven  Doppler 
Shift  Estimation  and  Compensation  Methods  for  Underwater  Wireless  Sensor  Networks”,  which 
is  to  analyze  and  develop  noncoherent  communication  methods  at  the  physical  layer  for  target 
tracking  and  use  the  information  theory  tools  to  predict  the  next  target  position  by  processing  the 
data  information  collected  from  a  collaborative  wireless  distributed  sensor  network  [4].  It 
improves  the  understanding  of  the  physical  communication  model  in  the  case  of  (2,  3)  DMCs.  In 
addition,  it  contributes  to  the  improvement  of  the  performance  limits  of  distributed  underwater 
wireless  sensor  networks,  by  maximizing  the  probability  of  target  detection  and  tracking  in  the 
presence  of  additive  Gaussian  noise.  Moreover,  we  compare  the  performance  of  the  different 
DMCs  for  better  signal  detection  in  underwater  wireless  sensors,  operating  with  limited 
transmitted  power  and  computational  environment.  The  results  of  this  work  will  contribute  to  the 
effort  of  overcoming  the  effect  of  noise  in  underwater  sensors  and  signal  processing. 

In  Chapter  3  we  derive  (33),  the  analytic  relationship  for  the  probability  of  error  in  the  (2, 
3)  DMC.  We  investigate  how  the  selection  of  the  threshold  with  respect  to  the  signal  amplitude 
affects  the  average  probability  of  error.  We  show  how  our  new  model  improves  signal  detection 
by  defining  symmetric  conditional  probabilities  around  the  zero  optimum  threshold  in  the  case  of 
the  (2,  3)  DMC. 

Our  findings  show  how  the  combination  of  the  (2,  2)  DMC  and  the  simple  symmetric  (2, 
3)  DMC  yields  a  better  signal  detection  in  the  presence  of  additive  Gaussian  noise.  In  the  case  of 
our  proposed  (2,  3)  DMC  for  the  underwater  distributed  wireless  sensor  networks,  the  probability 
associated  with  zero  optimum  crossing  threshold  remains  constant.  Thus  in  order  for  probability 
of  detection  to  be  maximized  by  minimizing  the  probability  of  false  alarm  and  miss,  the 
detection  scheme  needs  to  incorporate  suprathreshold  signal  and  a  symmetric  decision  threshold 
around  the  zero  optimum  threshold  in  the  case  of  (2,  2)  DMC. 


2 


Chapter  2 
Background 


In  this  chapter,  we  give  back  ground  review  on  concepts  used  throughout  the  thesis. 
Under  Water  Channel,  Probability  of  Error,  Threshold  deviees.  Discrete  Memoryless  Channels 
are  reviewed.  In  addition  we  discuss  Stochastic  Resonance  effect  in  nonlinear  systems. 

2.1  General  Communication  Systems  Model 

Electrieal  communication  systems  are  designed  to  send  messages  or  information  from  a 
source  that  generates  the  messages  to  one  or  more  destinations.  The  information  generated  by  the 
source  may  be  of  the  form  of  voiee  (speeeh  source),  a  picture  (image  source).  An  essential 
feature  of  any  source  that  generates  information  is  that  its  output  is  deseribed  in  probabilistie 
terms;  i.e.,  the  output  of  a  source  is  not  deterministic.  Otherwise,  there  would  be  no  need  to 
transmit  the  message.  The  structure  of  the  communieation  system  as  shown  in  Eigure  1  eonsists 
of  three  basic  parts,  namely,  the  transmitter,  the  ehannel  and  the  receiver.  The  funetions 
performed  by  these  three  elements  are  described  below. 

Transmitter:  The  transmitter  eonverts  the  electrical  signal  into  a  form  that  is  suitable  for 
transmission  through  the  physical  channel  or  transmission  medium. 

Receiver:  The  function  of  the  reeeiver  is  to  recover  the  message  signal  contained  in  the 
transmitted  signal.  If  the  message  signal  is  transmitted  by  carrier  modulation,  the  reeeiver 
performs  carrier  demodulation  in  order  to  extract  the  message  from  the  sinusoidal  carrier. 


Eigure  1:  General  communication  systems  model. 
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2.2  Communication  Channel 


A  communication  channel  refers  either  to  a  physieal  transmission  medium  sueh  as  a  wire, 
or  to  a  logieal  eonneetion  over  a  multiplexed  medium  sueh  as  a  radio  ehannel  as  shown  in  the 
figure  above.  A  ehannel  is  used  to  convey  an  information  signal,  for  example  a  digital  bit  stream, 
from  one  or  several  senders  (or  transmitters)  to  one  or  several  receivers.  A  ehannel  has  a  eertain 
eapaeity  for  transmitting  information,  often  measured  by  its  bandwidth  in  Hz  or  its  data  rate  in 
bits  per  seeond.  Depending  on  the  type  of  the  message  transmitted,  a  eommunieation  ehannel  is 
elassified  as  a  eontinuous,  discrete  and  mixed  channel. 

2.3  The  types  of  Signal  Propagation  in  Underwater 

There  are  three  types  of  transmission  medium  in  underwater  eommunieation  whieh  are 
aeoustie  eommunieation  (sound),  radio  wave  eommunieation  (eleetromagnetie)  and  optieal 
oommunieation  (light).  Radio  wave  and  optieal  eommunieation  does  not  work  well  in  deep 
underwater  environment.  Thus,  aeoustie  eommunieation  is  widely  ehosen  in  deep  underwater 
environment  due  to  the  low  attenuation  of  sound  in  water. 

Although  radio  wave  eommunieation  has  faster  propagation  speeds  eompared  to  the 
aeoustie  eommunieation,  there  are  limitation  faetors  when  using  eleetromagnetie  wave  in  deep 
underwater.  Eleetromagnetie  waves  strongly  attenuate,  espeeially  in  salt  water  and  propagating 
long  distanees  require  large  antenna  and  high  transmitter  power. 

Optieal  eommunieations  has  a  greater  advantage  in  data  rate  that  ean  exeeed  1  Giga  Hz. 
However,  when  used  in  underwater  environment,  the  light  is  rapidly  absorbed  and  the 
eommunieation  is  affeeted  by  its  seattering. 

2.4  Underwater  Acoustic  Channels 

In  reeent  years,  there  has  been  an  immense  interest  in  developing  underwater  aeoustie 
eommunieation  systems,  most  of  which  are  related  to  remote  eontrol  and  telemetry  applieations 
[2]. Other  applieations  include  ocean-bottom  survey  and  marine  monitoring,  eollection  of 
seientific  data  aequired  by  underwater  sensors  without  the  need  for  retrieving  the  equipment. 
However,  for  all  these  applieations  the  main  objeetive  is  to  aehieve  reliable  eommunieation  both 
in  point-to-point  links,  and  in  network  seenarios.  In  praetiee,  the  only  feasible  method  to  aehieve 
underwater  eommunieations  is  by  means  of  aeoustie  signals.  Sueh  aeoustie  links  are  exposed  to 
adverse  physieal  phenomena  governing  aeoustie  wave  propagation  in  the  sea.  These  inelude 
ambient  noise,  frequeney-dependent  attenuation,  temperature  and  pressure  variations, 
reverberation,  and  extended  multi-path.  Any  sueeessful  aeoustie  sensor  network  design  must 
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consider  all  these  effeets  in  order  to  select  an  appropriate  eonfiguration  for  system-related 
parameters.  The  transmit  power  level  and  operating  frequency  must  be  eonsidered  in  conjunction 
with  the  ambient  noise  and  transmission  range,  the  utilized  modulation  seheme,  data  rate,  and  the 
level  of  diversity  must  properly  mateh  the  expeeted  channel  eonditions  related  to  time  and 
frequency  dispersion. 

The  underwater  acoustic  communication  channel  is  extremely  complex  in  nature.  The 
eomplexity  arises  from  the  fact  that  the  ehannel  is  not  perfectly  homogeneous.  The  numerous 
imperfeetions  are  mainly  due  to  density  and  temperature  gradients  and  the  non-homogeneities  of 
the  water  due  to  suspended  partieles  of  solid  or  gaseous  matter.  The  constant  water  motion  and 
the  channel  boundaries  sueh  as  the  bottom  and  the  surface  further  increase  the  complexity.  Data 
in  UWC  suffers  from  multipath,  noise,  fading,  Doppler  spread,  and  high  and  variable 
propagation  delay.  The  factors  affecting  UWC  establish  temporal  and  spatial  variability  of  the 
acoustic  channel.  The  temporal  and  spatial  variability  makes  the  ehannel  very  limited  and  highly 
dependent  on  both  transmission  range  and  frequeney.  When  viewed  from  a  communieation 
system  designer’s  point  of  view,  the  four  aspeets  that  are  of  fundamental  coneern  are  namely, 
ambient  noise,  multi-path  phenomenon,  transmission  loss  and  Doppler  spreading  due  to  relative 
motion  between  transmitter  and  reeeiver. 


2.4.1  Ambient  Noise 

Ambient  noise  influences  the  received  signal  to  noise  ratio  and  largely  eontrols  the 
transmitter  power.  It  generally  deereases  in  frequency  over  the  range  of  interest.  Inshore 
environment  and  marine  worksites  are  noisier  than  deep  oeean  environment  and  the 
communication  system  design  needs  to  cater  for  the  worst  ease. 

For  band  limited  additive  white  Gaussian  noise  (AWGN),  mathematically  channel  capacity  is 
expresses  as 

C  —  BWlog2  (^1  +  bits/see  (1) 


where  ‘C’  is  channel  eapacity,  ‘51/F’  is  bandwidth,  ‘P’  is  the  power  of  signal  and  ‘Aq’  is  the 
power  of  noise.  Traditionally  engineers  have  been  interested  to  eliminate  the  effect  of  noise  in 
order  to  improve  system  performance.  In  our  work  we  investigated  the  effect  of  noise  in  order  to 
maximize  the  deteetion  of  the  reeeived  signal  by  employing  binary  input  ternary  output  diserete 
memoryless  ehannels. 
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2.4.2  Multi-path  phenomenon  causing  Inter-Symbol  Interference  and 
Reverberation 

This  is  the  most  challenging  aspect  of  the  underwater  acoustic  channel.  The  boundaries  of 
the  surface  and  bottom  reflect  the  energy;  so  numerous  travel  paths  exist  between  the  transmitter 
and  the  receiver.  This  is  further  complicated  by  imperfect  boundaries.  The  whole  phenomenon 
results  in  time  dispersal  of  the  signal.  This  time  spreading  can  be  as  high  as  hundreds  of 
milliseconds  in  shallow  water  to  several  seconds  in  deep  waters.  At  high  frequency,  the  total 
time  spread  is  less  due  to  absorption  at  boundaries  and  attenuation  in  water. 

2.4.3  Transmission  Loss  due  to  geometrical  spreading  and  absorption 

The  acoustic  wave  reduces  in  intensity  as  it  propagates  through  the  medium  due  to 
geometrical  spreading  and  absorption  mechanism.  Though  the  attenuation  of  acoustic  waves  in 
water  is  negligible  as  compared  to  the  radio  wave  in  water,  there  is  considerable  loss  in  energy 
due  to  absorption  mechanism  in  water.  The  loss  due  to  geometrical  spreading  can  be  either 
spherical  or  cylindrical  in  nature. 

Transmission  loss  (TL)  for  acoustic  communication  is  generally  expressed  as  the 
logarithmic  ratio  of  the  intensity  of  sound  at  the  reference  range  one  meter  away  from  the  center 
of  the  source  (b)  to  the  intensity  of  the  sound  at  the  receiver  (b).  TL  is  measured  in  decibels 
(dB).  Mathematically  it  can  be  expressed  as  follows: 

TL  =  lOlog(l^)  (2) 

2.4.4  Doppler  spreading  due  to  relative  motion  between  transmitter  and 
receiver 

This  is  introduced  by  relative  motion  between  the  transmitter  and  receiver,  or  by  motion 
of  the  water.  Due  to  the  scarcity  of  channel  bandwidth  (caused  by  absorption  losses  and  projector 
transducer  characteristics),  Doppler  spread  may  easily  result  in  further  reduction  in  available 
bandwidth. 

Due  to  harsh  underwater  acoustic  medium  described  above,  many  well-known 
communication  techniques  cannot  be  applied  and  hence  the  design  is  a  compromise  between  data 
rate,  reliability  and  distance.  The  combined  effects  of  the  above  mentioned  phenomenon  have  led 
in  the  past  to  system  design  based  exclusively  on  noncoherent  detection  and  low  signaling  rates. 
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Approaches  to  system  design  depend  upon  the  teehnique  used  for  overeoming  the  effeets 
of  multi-path  and  time  variations: 

■  the  signal  design,  i.e.,  the  ehoiee  of  modulation  /  deteetion  method 

■  the  transmitter  /  reeeiver  strueture 


2.5  Discrete  Data  Transmission 

Diserete  data  transmission  as  shown  in  figure  below  is  the  transmission  of  one  message 
from  a  finite  set  of  messages  through  a  eommunieation  ehannel.  A  message  sender  at  the 
transmitter  eommunieates  with  a  message  reeeiver.  The  sender  seleets  one  message  from  the 
finite  set,  and  the  transmitter  sends  a  eorresponding  signal  (or  waveform)  that  represents  this 
message  through  the  eommunieation  ehannel.  The  reeeiver  deeides  the  message  sent  by 
observing  the  ehannel  output.  Sueeessive  transmission  of  diserete  data  messages  is  known  as 
digital  eommunieation.  Based  on  the  noisy  reeeived  signal  at  the  ehannel  output,  the  reeeiver 
uses  a  proeedure  known  as  deteetion  to  deeide  whieh  message,  or  sequenee  of  messages,  was 
sent.  Optimum  deteetion  minimizes  the  probability  of  an  erroneous  reeeiver  deeision  on  whieh 
message  was  transmitted. 

The  ehannel  distorts  the  transmitted  signals  both  deterministieally  and  with  random  noise. 
The  noisy  ehannel  output  will  usually  not  equal  the  ehannel  input  and  will  be  deseribed  only  in 
terms  of  eonditional  probabilities  of  various  ehannel-output  signals.  The  ehannel-input  signals 
have  probabilities  equal  to  the  probabilities  of  the  messages  that  they  represent.  The  optimum 
deteetor  will  depend  only  on  the  probabilistie  model  for  the  ehannel  and  the  probability 
distribution  of  the  messages  at  the  ehannel  input.  The  general  optimum  deteetor  speeializes  in 
many  important  praetieal  oases  of  interest. 


Estimate  of 


Figure  2:  Blook  diagram  of  diserete  data  transmission. 


2.5.1  Discrete  Memoryless  Channels 

The  fundamental  model  desoribing  eommunieation  over  a  noisy  ehannel  is  the  so-oalled 
diserete  memoryless  ehannel  (DMC).  A  DMC  oonsists  of : 
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•  a  finite  input  alphabet  described  by  the  random  variable  x 

•  a  finite  output  alphabet  described  by  the  random  variable  y 

•  a  conditional  probability  distribution  P(y|x) 

For  a  memoryless  channel,  the  output  depends  on  the  input  Xj  at  the  same  time  i.  Therefore, 
the  conditional  probability  distribution  is  given  by 


P{y\x)  ^\YiP(yi\Xi)  (3) 

where  n  is  the  number  of  symbols  in  the  alphabet. 


Input  alphabet  X  Output  alphabet  Y 


Figure  3:  Discrete  memoryless  channel. 

2.5.2  DMC  General  Model  (M,  N) 

The  input  of  a  DMC  is  random  variable  x,  who  selects  its  value  from  a  limited  discrete 
set  X,  as  shown  in  the  Figure  4.  All  the  transition  probabilities  from  Xj  to  yj  are  gathered  in  a 
transition  matrix  p(y|x).The  (1,7)  entry  of  the  matrix  is  P(y=  yj\X  —  Xj)  which  is  called  forward 
transition  probability. 


Figure  4;  DMC  general  model. 
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The  general  DMC  is  called  memoryless  because  the  current  output  yj  depends  only  on  the 
current  input  i. 

The  capacity  of  DMC,  p(y\x)  is  defined  by 

C  =  maXp^x^I(_X;Y)  (4) 

where  I(X-,Y)  is  the  mutual  information  of  the  two  discrete  random  variables  X  and  Y  given  by 
[19] 

KX-,  Y)  =  Y.y„Y.„^P{x,y)  log  (4.1) 

and  the  maximum  is  taken  over  all  input  distribution-P(x). 

2.5.3  DMC  I 

Let  X  and  Y  be  discrete  alphabets  and  P(y  |x)  be  a  transition  probability  from  X  to  Y.  A 
discrete  channel  P(y  |x)  is  single-input  single-output  with  input  random  variable  X  taking  values 
in  X  and  output  random  variable  Y  as  shown  in  Figure  5  taking  values  in  Y  such  that 

P^{X  =  x,Y  =  y]  =  PriX  =  x}P(y|x)  (5.1) 

where  Pj.(. )  is  the  probability  of  a  random  variable. 


Figure  5:  DMC  I. 

2.5.4  DMC  II 

Let  Xi  Y,  and  2^  be  discrete  alphabets,  and  P(y|x)  be  a  transition  matrix.  Let  a:  X  ^ 

Y  and  Z  be  a  random  variable  taking  values  in  A  discrete  channel  (a,  Z)  is  a  single-input 
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single-output  system  with  input  X  and  output  Y  as  shown  in  Figure  6.  For  any  input  random 
variable  x,  the  noise  variable  z  is  independent  of  x,  and  the  output  random  variable  Y  is  given  by 

Y  =  a(X,Z)  (5.2) 

Where  (a,  Z)  is  a  sequence  of  replicates  of  generic  discrete  channel  (a,  Z) 

Xi  and  Yi  respectively  represent  the  input  and  output  of  the  DMC  at  time  i. 


X 


a 

>> 

/ 

Figure  6:  DMC  II. 


2.5.5  Example  of  DMC;  binary  -input  binary-output  (2,  2)  DMC 

A  binary  symmetric  channel  (or  BSC)  as  shown  in  Figure  7  is  a  common 
communications  channel  model  used  in  coding  theory  and  information  theory.  In  this  model,  a 
transmitter  wishes  to  send  a  bit  (a  zero  or  a  one),  and  the  receiver  receives  a  bit.  It  is  assumed 
that  the  bit  is  usually  transmitted  correctly,  but  that  it  will  be  "flipped"  with  a  small  probability 

(the  "crossover  probabihty")p  <  -.  This  channel  is  used  frequently  in  information  theory  because 
it  is  one  of  the  simplest  channels  to  analyze. 
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There  is  no  feedback  link  from  the  receiver  back  to  the  transmitter  that  would  inform  the 
transmitter  about  the  last  outputs.  The  transition  probabilities  from  X  toY  are  given  below 


1  -  p  =  Pr(Y  =  0\X  =  0)  =  Pr(Y  =  1|X  =  1) 


(6.1) 


where  Pf.(Y  =  y\X  =  x)  represents  the  probability  of  receiving  y  given  the  signal  x  was 
transmitted.  Similarly 


p  =  p^(Y  =  0\X  =  1)  =  Pr(Y  =  1\X  =  0) 


(6.2) 


The  channel  transition  matrix,  M  of  the  BSC  is  given  by 

1%) 

The  capacity  ’C’  of  the  channel  can  be  calculated  to  be  [19] 

C  =  1  +  p  X  log2  p  +  (1  —  p)  X  log2(l  —  p)  Bits/transmission.  (6.4) 

The  binary  symmetric  channel  is  one  of  the  simplest  noisy  channels  to  analyze.  Many  problems 
in  communication  theory  can  be  reduced  to  a  BSC.  Conversely,  being  able  to  transmit  effectively 
over  the  BSC  can  give  rise  to  solutions  for  more  complicated  channels. 


2.5.6  Additive  White  Gaussian  Noise  Channel  (AWGN) 

Perhaps  the  most  important,  and  certainly  the  most  analyzed,  digital  communication 
channel  is  the  AWGN  channel  shown  in  Figure  8.  This  channel  passes  the  sum  of  the  modulated 
signal  A(t)  and  additive  Gaussian  noise  iV(t)  to  the  output.  The  Gaussian  noise  is  assumed  to  be 
uncorrelated  with  itself  (or  “white”  )  for  any  non-zero  time  offset. 


Xit) 
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This  is  a  time  discrete-channel  with  output  Tj  at  time  i,  where  Tj  is  the  sum  of  the  input 
Xi  and  the  noise  iVj.  The  noise  iVj  is  drawn  i.i.d.  with  a  Gaussian  distribution  with  variance  cr^  . 

Yi^  Xi+  Ni  where  iVj~  (0,  a^)  (6.5) 


Figure  9;  AWGN,  iV~  (0,  1). 

Figure  9  shows  a  Gaussian  noise  with  zero  mean  and  variance  1 . 

2.6  Bit  Error  Rate  (BER) 


In  digital  transmission,  the  number  of  bit  errors  is  the  number  of  received  bits  of  a  data 
stream  over  a  communication  channel  that  has  been  altered  due  to  noise,  interference,  distortion 
or  bit  synchronization  errors.  In  our  study  of  Discrete  Memoryless  Channels  (DMC)  in  Chapter 
3  we  used  the  average  bit  error  rate  as  performance  measure  to  investigate  the  (2,3)  DMC  in  the 
presence  of  Additive  Gaussian  noise. 

2.6.1  Factors  affecting  bit  error  rate 

In  a  communication  system,  the  receiver  side  BER  may  be  affected  by: 

>  transmission  channel  noise, 

>  interference,  distortion, 

>  bit  synchronization  problems, 

>  Attenuation,  wireless  multipath  fading,  etc. 
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The  BER  may  be  improved  by  ehoosing  a  strong  signal  strength  (unless  this  eauses  eross- 
talk  and  more  bit  errors),  by  ehoosing  a  slow  and  robust  modulation  seheme  or  line  eoding 
seheme,  and  by  applying  ehannel  eoding  sehemes  sueh  as  redundant  forward  error  eorreetion 
eodes. 

The  transmission  BER  is  the  number  of  deteeted  bits  that  are  ineorreet  before  error 
eorreetion,  divided  by  the  total  number  of  transferred  bits  (ineluding  redundant  error  eodes). The 
information  BER,  approximately  equal  to  the  deeoding  error  probability,  is  the  number  of 
deeoded  bits  that  remain  ineorreet  after  the  error  eorreetion,  divided  by  the  total  number  of 
deeoded  bits  (the  useful  information).  Normally  the  transmission  BER  is  larger  than  the 
information  BER. 

2.6.2  Hypothesis  testing 

In  statisties,  a  type  I  error  (or  error  of  the  first  kind)  is  the  ineorreet  rejeetion  of  a  true  null 
hypothesis.  A  type  II  error  (or  error  of  the  seeond  kind)  is  the  failure  to  rejeet  a  false  null 
hypothesis.  A  type  I  error  is  a  false  positive.  Usually  a  type  I  error  leads  one  to  eonelude  that  a 
supposed  effeet  or  relationship  exists  when  in  faet  it  doesn't.  In  digital  eommunieation  systems 
type  I  and  type  II  errors  are  analogous  to  probabilities  of  false  alarm  and  miss  respeetively. 

In  underwater  distributed  wireless  sensor  networks  where  target  deteetion  and  traeking  is 
so  important  [4],  the  probability  of  false  alarm  and  deteetion  plays  an  important  role.  The  first 
hypothesis  is  denoted  as  the  null  hypothesis  Hq  and  the  seeond  as/f^.  The  deteetion  logie 
therefore  must  examine  eaeh  sensor  measurement  to  be  tested  and  seleet  one  of  the  hypotheses  as 
“best”  aeeounting  for  that  measurement.  If  Hq  best  aeeounts  for  the  data,  the  system  deelares 
that  a  target  was  not  present  at  the  range,  angle,  or  Doppler  eoordinates  of  that  measurement;  if 
best  aeeounts  for  the  data,  the  system  deelares  that  a  target  was  present. 

The  analysis  starts  with  a  statistieal  deseription  of  the  probability  density  funetion  (pdf) 
that  deseribes  the  measurement  to  be  tested  under  eaeh  of  the  two  hypotheses.  If  the  sample  to  be 
tested  is  denoted  as  y,  then  fiylS-J  and  f(y\Si)  are  required  to  model  the  deteetion  seheme  as 
shown  in  Eigure  10.  The  binary  digits  0  and  1  are  represented  by  voltage  waveforms  Sq  and  5^ 
respeetively. 
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Figure  10:  Zero  erossing  deteetion  seheme. 

As  shown  in  figure  above  the  deteetion  problem  is  to  develop  models  for  these  two  pdfs. 
In  fact,  analysis  of  underwater  distributed  wireless  sensor  networks  performance  is  dependent 
on  estimating  these  pdfs  for  the  system  and  scenario  at  hand.  Furthermore,  a  good  deal  of  the  (2, 
3)  DMC  system  design  problem  is  aimed  at  manipulating  these  two  pdfs  in  the  presence  of 
additive  white  Gaussian  noise  in  order  to  obtain  the  most  favorable  detection  performance  by 
minimizing  probability  of  error. 

2.7  Different  probabilities  of  error 

Once  the  two  pdfs  are  successfully  modeled,  the  following  probabilities  of  interest  can  be 
defined:  probabilities  of  detection,  false  alarm,  miss  and  rejection. 

2.7.1  Probability  of  true  detection  (  P^f) 


The  probability  that  a  target  is  declared  (i.e.,  5^  is  chosen)  when  a  target  is  in  fact  present. 
This  is  the  area  (5i  +  52)  by  which  the  pdf,  /(y|5’i)  is  greater  than  the  decision  threshold  0  as 
shown  in  Figure  10. 

'’«=  r/(y|Si)dy  (7) 


where  f(y)  is  the  probability  density  function  of  the  normally  distributed  Gaussian  random 
variable  with  mean  p,  and  variance  cr^ 


f(y\s)  = 


1 

V  27ro'2 


-(y-A)^ 
X  e  20-2 


(8) 


Thus  the  probability  of  detection  can  be  reduced  into 
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(9) 


Pdt~ 


/, 


0  4~2 


-(y-Mr 


dy 


2.1.2  Probability  of  false  alarm  (  Pfa) 


The  probability  that  a  target  is  declared  (i.e.,  5^  is  chosen)  when  a  target  is  in  fact  not 
present.  It  is  the  area  {B2)  by  which  the  Pdf,  /(y|5’o)  is  greater  than  the  decision  threshold  0  as 
shown  in  Figure  10. 

Pfa^  C  f(y\So)dy  (10) 

Using  (8)  the  probability  of  false  alarm  is  mathematically  defined  as 

„  .00  1  -(y-Apf 

fo  “y  (“> 

2.7.3  Probability  of  miss  (Pm) 


The  probability  that  a  target  is  not  declared  (i.e.,  Sq  is  chosen)  when  a  target  is  in  fact 
present.  This  is  the  area  (B2)  by  which  the  Pdf,/(y|5i)  is  less  than  the  decision  threshold  0  as 
shown  in  Figure  10. 

2.7.4  Probability  of  rejection  (P^) 

The  probability  that  a  target  is  not  declared  (i.e.,  Sq  is  chosen)  when  a  target  is  not 
present.  This  is  the  area  (B3  +  B^)  by  which  the  Pdf,/(y|5o)  is  less  than  the  decision  threshold 
theta  as  shown  in  figure  above. 

Note  that  =  1  -  and  Pj.  =l-P/a  .  Thus,  P^t  and  Pfa  are  suffice  to  specify  all  of  the 
probabilities  of  interest.  As  the  latter  two  definitions  imply,  it  is  important  to  realize  that,  because 
the  problem  is  statistical,  there  will  be  a  finite  probability  that  the  decisions  will  be  wrong.  Thus 
by  adjusting  our  threshold  0  and  the  noise  power  a,  the  (2,  3)  DMC  can  achieve  a  better 
performance  in  minimizing  the  average  probability  of  error. 

2.8  Binary-input  binary  output  DMC  and  average  probability  of  error 


The  (2,  2)  DMC  is  shown  in  Figure  1 1.  It  is  characterized  by  the  input  random  variable  X 
and  the  output  random  variable  Y,  taking  values  (Xi,X2)  and  (yi,y2),  respectively.  Its  transition 
probability  matrix  M  is  defined  by 
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(12.1) 


a 

1 

—  a 

a 

a 

1 

-P. 

fi 

P. 

where  a  and  P  are  defined  as: 


a  =  P(r  =  =  Xi),  p=P(Y^  y^lX  =  x^) 


(12.2) 


Figure  1 1 :  Binary  asymmetrie  ehannel. 

For  the  Binary  input-binary  output  asymmetrie  ehannel,  a  simple  threshold  based 
deteetion  deteetor  ean  be  employed  as  shown  in  the  Figure  12  to  study  its  performanee  and 
investigate  the  effeet  of  noise,  whieh  eould  be  a  eorner  stone  in  the  diseussion  of  the  (2,  3) 
DMC  that  will  follow. 

In  the  (2,  2)  DMC  the  souree  maps  the  bits  0  and  1  into  voltage  levels  -A  and  A 
respeetively  as  shown  in  Figure  11.  The  massage  is  transmitted  aeross  the  ehannel  where  there  is 
an  additive  white  Gaussian  noise.  The  deteetor  with  threshold  0  after  the  observation  of  the 
transition  probabilities  deeides  whieh  bit  was  transmitted  with  average  probability  of  error  Pg  or 
average  probability  of  deteetion 

The  deteetor  employes  a  deeision  rule  by  whieh  if  the  signal  is  greater  than  the  threshold 
0,  it  deelares  messgae  5^  was  transmitted,otherwise  Sq  with  eertain  probability  of  error. 
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Figure  12:  (2,  2)  DMC  with  simple  detection  employed. 


The  average  probability  of  error  of  the  (2,  2)  DMC  can  be  evaluated  using  the  concept  of 
probabilities  of  detection  and  false  alarm  as  such: 

P(e|5o)=  Pr(Y  >  6)  =the  probability  of  error  associated  with  bit  0 

Using  (10)  and  (11),  the  probability  that  we  declare  a  target  is  present  when  in  fact  is  not  present 
is  given 

,.00  -(y+^P 

P(.e\So=)  =  Pfa=  Iq  7^^®  (13) 

For  a  Gaussian  random  variable,  iV~(/r,  cr^),  a  simple  change  of  variable  in  the  integral  in  order 
to  compute  Pf.(Y  >  y),  results  in 


PAY  >y)  =  Q  (1^) 

Where  (2(x)is  the  error  function  defined  as 


X  e~ dx 


(14) 


(15) 


As  the  <2(x)  is  symmetric,  the  following  relationships  can  be  also  defined  mathematically 


Q(-x)  =  1  -  Qix),  QAA  -  QCxz)  =  47^  X  e  2  dx  (16) 

Using  (14),  the  probability  of  false  alarm  is  given 

P(ie\So)  =  Q{^)  (17) 

Similarly  the  probability  that  we  miss  a  target  when  it  is  in  fact  present,  P(e|5i)  =  Pj.(T  <  6) 
=the  probability  of  error  associated  with  bit  1 . 

Using  (9)  and  (14),  the  probability  of  miss  is  given  by 
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(18) 


P(e|5i)  =  l-Q(^) 

The  average  probability  of  error  as  a  funetion  of  the  threshold  0  ean  be  expressed  [18] 

P(e|e)  =  Po  X  P(e|5o)  +  Pi  x  P(e|5i)  (19) 

where  Po  &  Pithe  prior  probability  of  the  bit  0  and  1  respeetively 
Thus, 


P(e|e)  =  Po  X  Q  (^)  +  p^x(l-Q  (^)) 

Average  probability  of  deteetion,  P(d|6)  =  1  —  j^po  x  Q  (“■^)  +  Pi  ^  (“■^)) 


(20) 

(20.1) 


The  average  probability  of  error  is  a  funetion  of  signal  amplitude,  the  decision  threshold  0  and 
the  noise  power  a. 


Figure  13:  Probability  of  detection  vs  a. 

As  shown  in  the  Figures  13-14  using  (20.1)  and  (20),  at  the  optimum  value  of  the  noise 
power  a,  the  detector  exhibits  few  errors  as  the  probability  of  correct  detection  is  maximized.  A 
further  increase  of  the  noise  power  beyond  that  optimum  value  of  noise  deteriorates  the  detection 
performance  of  the  receiver  in  line  with  common  sense. 
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Figure  14;  (2,  2)  DMC  performance  with  the  AWGN.  The  solid  and  dotted  lines  represent  the 
average  probabilities  of  detection  and  error,  respectively. 

Beyond  the  optimum  value  a,  the  performance  of  the  (2,  2)  DMC  as  shown  in  Figure  14 
deteriorates  and  the  addition  of  noise  becomes  disadvantageous.  It  is  important  to  note  that,  as 
the  decision  threshold  0  increase  far  away  from  the  signal  amplitude  A  the  performance  of  the 
detector  decreases.  Increasing  the  decision  threshold  from  1.1  to  1.3  for  instance  as  shown  in  the 
Figure  13,  there  is  a  remarkable  increase  in  the  average  probability  of  error  at  the  crossing  of  the 
optimum  noise  power,  sigma.  Thus,  the  optimum  value  of  the  noise  power  a  is  also  a  function  of 
the  threshold  0  for  optimum  system  performance. 

2.9  Stochastic  Resonance 

In  general,  the  SR  phenomenon  is  a  non-linear  effect  where  in  communication  systems 
the  transmission  of  the  information  is  enhanced  in  the  presence  of  the  additive  noise  [5].  Various 
performance  metrics  in  the  presence  of  SR  such  as  signal  to  noise  ratio,  mutual  information,  and 
channel  capacity  improvement,  under  a  certain  range  of  power  noise  levels,  were  discussed  in 
[3],  [5], [7], [8]  and  [13].  In  Figure  15,  the  average  probability  of  error  is  plotted  as  a  function  of 
the  noise  power  level.  It  is  observed  that  the  average  probability  of  error  decreases,  as  the  noise 
level  increases,  until  the  detection  performance  reaches  a  resonance  peak.  This  decrease  in  the 
probability  of  error  is  not  in  line  with  the  common  intuition  that  increasing  the  noise  power  level 
increases  probability  of  error. 
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Figure  15:  Probability  of  error  in  (2,  2)  DMC. 

In  this  chapter,  we  gave  back  ground  review  on  concepts  used  throughout  the  thesis. 
Under  Water  Channel,  Probability  of  Error,  Threshold  Devices,  Discrete  Memoryless  Channels 
are  reviewed.  We  also  modeled  the  average  probability  of  error  in  the  (2,  2)  DMC  as  Q  function. 
Utilizing  this  analytic  relationship,  we  discussed  the  performance  of  the  (2,  2)  DMC  in  the 
presence  of  additive  Gaussian  noise.  Finally  we  discussed  Stochastic  Resonance  effect  in 
nonlinear  systems,  in  particular  in  the  case  of  the  (2,  2)  DMC. 
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Chapter  3 

(2,  3)  DMC  and  Probability  of  Error 


In  this  chapter,  we  focus  on  the  general  (2,  3)  DMC  and  its  particular  case,  the  (2,  2)  binary 
asymmetrie  ehannel.  In  Seetion  3.2,  we  derive  the  average  probability  of  error  for  the  (2,  3) 
DMC. 


3.1  The  (2,  3)  DMC 


The  (2,  3)  DMC  {M:  X  ^  T}  is  eharacterized  by  a  binary  input  random  variable  X  =  {1,-1}, 
a  ternary  output  random  variable  Y  =  {1,0,  —1}  and  a  transition  probability  matrix,  M,  [5]. 


M  = 


Pzi  P3l\ 
P22  P32) 


where  Pji,  is  the  probability  of  receiving  signal  j  given  signal  i  was  transmitted.  The  transition 
probability  matrix  from  X  to  T  is  given  by 


Pii  =  P(Y  =  1\X  =  1)  ,P2i  =  P(T  =  0|X  =  l),P3i  =  P(T  =  -1\X  =  1) 
P12  =  P(Y  =  1\X  =  -1),P22  =  P(Y  =  0\X  =  -1),P32  =  P(Y  =  -1\X  =  1) 


Figure  16:  (2,  3)  DMC  representation. 
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The  input  to  the  threshold  communication  channel  is  the  signal  that  takes  the  binary  values  ±A 
as  in  [5],  with  probability  po  andpi.  The  physical  model  is  represented  in  Figure  16. The  (2,  3) 
DMC  detector  transforms  the  observation  into  a  value  which  is  finally  compared  to  a  threshold  0 
to  make  a  decision. 


3.2  System  Model 

The  input  to  the  threshold  communication  channel  is  the  signal  that  takes  the  binary  values 
±A  as  in  [5],  [14]  with  probability  po  and  pi.  The  physical  model  is  represented  in  Figure  17.  The 
(2,  3)  DMC  detector  transforms  the  observation  into  a  value  which  is  finally  compared  to  a 
threshold  0  to  make  a  decision.  The  DMC  is  completely  characterized  by  the  transition 
probabilities  of  the  output  conditioned  by  the  input  probabilities. 


?  t(0) 


>T(t)  = 


{ 


0 

-0 


/V(t)~(0,a2) 

Figure  17:  System  model  of  (2,  3)  DMC. 

The  physical  communication  channel  contains  a  threshold  decision  block  function 
threshold  t(6),  where  t(6)  represents  the  threshold  value.  Based  on  the  threshold  level  0  and 
the  noise  level,  the  received  signal  is  converted  into  a  discrete  random  variable  Y (t)  taking  on 
the  values  of  1,  0,  or  -1.  We  assumed  antipodal  signaling  and  a  new  symmetric  decision 
threshold  was  defined  around  the  optimum  threshold  in  the  (2,  2)  DMC  which  gave  rise  into 
three  distinct  regions  at  the  output  as  shown  in  figure  above. 

Adapting  the  model  of  [5],  we  first  defined  four  different  regions  of  interest.  These  are 
Cq,  Cl  Dq  and  as  shown  in  Figure  18.  We  incorporated  the  (2,  2)  and  the  classical  simple  (2,  3) 
DMCs  optimum  detection  schemes  to  study  the  effect  of  additive  Gaussian  noise  in  the  case  of 
our  modified  (2,  3)  DMC.  The  (2,  2)  and  (2,  3)  DMCs  are  represented  with  zero  detection 
threshold  and  symmetric  threshold  0  respectively. 
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Finally  we  derived  the  analytie  relationship  for  the  probability  of  error  in  the  (2,  3)  DMC.  We 
investigated  the  effeet  of  AWGN  on  the  probability  of  error  of  the  (2,  3)  DMC  for  different  value 
of  deeision  threshold  0  and  noise  power  a. 


Figure  18:  Regions  of  interest  in  the  (2,  3)  DMC.  Bit  0  and  1  are  represented  by  voltage 
waveforms  Sq  and  5^  respeetively.  A  is  the  signal  amplitude  and  0  our  symmetrie  deeision 
threshold. 

We  defined  probability  of  miss  as  the  eonditional  probability  that  the  reeeived  signal  is  less  than 
the  threshold  0  given  it  is  greater  than  the  zero  optimum  threshold  in  the  (2,  2)  DMC.  Similarly 
the  probability  of  false  alarm  is  the  eonditional  probability  that  the  reeeived  signal  is  greater  than 
the  symmetric  negative  threshold  theta  given  that  the  received  signal  is  less  than  the  optimum 
threshold  0  in  the  (2,  2)  DMC.  Thus,  we  have  the  following  relations. 

P(e|5i)  =  P(Di|Ci)=^^^  (21) 

where  is  the  region  for  which  the  conditional  probability  f(,y\S^)  is  less  than  the  threshold  0 
as  shown  in  Figure  18.  Also  is  the  region  for  which  the  conditional  probability /(y|5i)  is 
greater  than  the  zero  decision  threshold  as  shown  in  figure  above. 

These  two  areas  are  intersecting  in  a  certain  fashion  as  follows: 

p(CinDi)  =  J®/(yl‘?i)dy  (22) 

Using  (8),  (14),  and  (15),  the  area  of  intersection  can  be  modeled  as  Q  function 

P(Ci  nD,)  =  l-Q[f)-Q  (^)  (23) 

The  detector  correctly  decodes  the  bit  1  if  it  receives  a  voltage  greater  than  zero  in  the  (2,  2) 
DMC.  Therefore, 
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(24) 


P(iC^)  =  Jo“/(y)dy 
Substituting  (8)  and  (14),  we  obtain 

p(Ci)  =  1  -  Q  (^) 
Combining  (22)  and  (25)  we  will  get 


P(e|5i) 


Following  similar  argument  the  probability  that  is  assoeiated  with  bit  zero  ean  be  ealeulated  as 
follows 


P(e|5o)  =  PCPoKo)  = 


P(ConDo) 


where  Dq  is  the  region  for  whieh  the  eonditional  probability  /(yIS’o)  is  greater  than  the  -0 
threshold  as  shown  in  Figure  18.  Also  Cq  is  the  region  for  whieh  the  eonditional 
probability  /(y  |5’o)  is  less  than  the  zero  deeision  threshold  . 

The  area  for  whieh  these  parameters  of  interest  are  overlapping  ean  be  summarized  as  follows: 

P(Con£lo)-J_V(yl^o)dy  (28) 

Employing  (8),  (14),  and  (15)  we  will  obtain 

PiConDo)  =  Q{^)-Q{^)  (29) 

Also  a  simple  (2,  2)  DMC  decodes  the  bit  zero,  when  the  voltage  received  is  less  than  the 
reference  zero  voltage. 


Therefore, 


P(Co)  =  l°f(y\S„)dy 


Using  (8),  (14),  and  (15)  we  will  obtain 


p(Co)  =  1  -  <2  © 


Combining  (28)  and  (30),  we  get  the  probability  of  false  alarm  that  the  detector  declares  a  target 
is  present  when  in  fact  is  missing  is  represented  by  the  following  Q  function 


P(e|5o)  = 
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Using  (19)  the  average  probability  of  error  for  the  (2,  3)  DMC  whieh  analytically  incorporates 
the  optimum  detection  threshold  of  the  (2,  2)  DMC,  will  be  calculated  as  such 


P(e|e)  =  Po  X  +  Pi  X 


Since  probability  of  error  and  detection  add  up  to  1.  We  get, 
P(d\e)  =  1 


(33) 


(33.1) 


where  pg  and  pi  are  the  prior  probabilities  of  bit  0  and  1  respectively  and  P(d)  is 
probability  of  detection. 


As  the  classical  probability  of  error  curves  are  plotted  with  respect  to  the  signal  to  noise 
ratio  (y),  we  also  remodeled  our  average  probability  of  error  for  the  (2,  3)  DMC  by  letting 
^  =  Vk  •  By  doing  so,  we  were  able  to  compare  (32)  with  conventional  probability  of  error 

curves  and  deduce  the  location  of  our  symmetric  decision  threshold  relative  to  the  signal 
amplitude. 


P(e|e)  =  Po  X 


'-I 


i-QCVr) 


))-(2(Vy)  ^  i-(2(Vy)(l-i)-(2(Vy) 

+  Pi  X 


i-QCVr) 


(34.1) 


As  (33)  shows  the  average  probability  of  error  is  a  function  of  signal  amplitude,  noise 
intensity  and  decision  threshold  0.  Using  this  analytic  relationship  we  studied  how  the 
probability  of  error  varies  as  a  function  of  signal  to  noise  ratio  y  by  either  varying  the  signal 
amplitude  and  fixing  the  threshold  or  vice  versa.  We  examined  how  the  amplitude  of  the  signal  is 
related  to  the  decision  threshold  0  by  plotting  the  probability  of  error  curves  as  a  function  of  y  as 
shown  in  Figures  19-20. 
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Probability  of  etror 


Figure  19:  Probability  of  error  vs  y  when  0  is  0.75. 

In  order  to  achieve  the  same  probability  of  error  by  either  fixing  the  threshold  or  the 
signal  amplitude,  one  needs  to  increase  the  y  to  compensate  the  decrease  in  the  Euclidian 
distance  between  the  threshold  and  signal  amplitude  as  shown  in  the  Figure  above.  Alternatively 
speaking  for  the  same  y  the  probability  of  error  increase  as  the  threshold  approaches  the 
amplitude  of  the  signal. 


Y 

Figure  20:  Probability  of  error  vs  y  for  signal  amplitude,  A=l. 
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As  shown  in  Figure  20,  for  the  same  y,  the  probability  of  error  increases  as  the  detection 
threshold  0  increases  and  approaches  our  signal  amplitude  (A=l).  For  the  same  value  of  0,  the 
probability  of  error  decreases  as  the  y  increases. 


Figure  21;  Probability  of  error  for  the  (2,  3)  DMC  vs  y  for  0=1.2. 

However,  if  threshold  becomes  >  the  amplitude  of  the  signal  as  shown  in  the  Figure  21,  then 
we  end  up  of  making  a  total  error,  i.e.  for  A=1  and  1.1  in  figure  above.  Thus,  to  achieve  a 
minimum  probability  of  error,  the  threshold  0  should  be  less  than  our  signal  amplitude  in  our 
modified  (2,  3)  DMC,  i.e  O<0<A. 

Alternatively,  if  we  increase  the  Euclidian  distance  between  the  signal  amplitude  and  the 
threshold  beyond  a  certain  limit,  i.e.  A>  0  the  probability  of  error  curves  become  close  and  fall 
sharply.  The  minimum  attainable  probability  of  error  is  obtained  at  lower  y. 
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Figure  22:  Probability  of  error  for  the  (2,  3)  DMC  as  a  funetion  of  y  for  0=0.75. 

As  depict  in  figure  above,  as  our  detection  threshold  theta  approximately  becomes  < 

the  probability  of  error  curves  decay  so  fast  compared  to  Figure  20-21.  Also  for  the  same  y,  the 
probability  of  detection  gain  among  the  different  signaling  scenarios  become  negligible. 

After  studying  the  effect  of  the  relative  Euclidian  distances  between  the  signal  amplitude 
and  the  detection  threshold  0  on  the  average  probability  of  error  in  our  modified  (2,  3)  DMC,  we 
proceed  our  investigations  to  study  the  effect  of  the  above  observed  dependencies  considering 
probabilities  of  true  detection,  false  alarm,  miss  and  rejection  in  the  modified  (2,  3)  DMC. 


Probability  of  false  alarm  in  the  modified  (2,  3)  DMC  is  nothing  but  P(e|5o).  Since  probability 
of  false  alarm  and  rejection  add  up  into  one,  using  (32)  we  get  (34.2)  as  follows: 


Pie\So)  =  Pfa  = 


-<20) 


,  and  Pf 


=  1  - 


<2(0^)-<20) 

-<20) 


(34.2) 


where  Pfa  and  Py  are  the  probabilities  of  false  alarm  and  rejection  respectively. 


Similarly  as  the  probability  of  miss  and  true  detection  sum  up  into  one.  Utilizing  the  fact  that  the 
probability  of  miss  is  equivalent  to  P(e|5i),  we  arrived  at  (34.3)  as  follows: 


l_Q(d)  _  Q(2zd) 

P(e|5i)  =P^=  \  ^  ,and 

-<2y 


-<20)-  <2(0^) 

-<20) 


(34.3) 


where  P^  and  P^t  are  the  probabilities  of  miss  and  true  detection  respectively. 
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Using  (34.2)  and  (34.3)  as  shown  in  Figures  23-27,  we  studied  how  the  selection  of  the  threshold 
0,  the  signal  amplitude  and  the  noise  power  determine  these  above  probabilities  of  interest. 


Figure  23:  Probabilities  of  error  vs  0.  The  signal  amplitude  is  1. 

The  selection  of  the  decision  threshold  0,  not  only  affects  the  average  probability  of  error 
but  also  the  probabilities  of  false  alarm,  true  detection,  miss  and  rejection.  Analytic  relationships 
(33.2)  and  (33.3)  are  used  to  study  the  above  mentioned  probabilities.  As  the  threshold  0 
increases  towards  the  signal  amplitude,  as  shown  in  the  Figure  23,  the  probabilities  of  false  alarm 
and  miss  increase.  Also  the  probabilities  of  true  detection  and  rejection  sharply  decrease  with 
increase  of  0.  After  observing  the  effect  of  0  on  the  above  mentioned  probabilities,  we 
conducted  similar  investigations  by  corresponding  each  of  the  probabilities  against  the  other  as 
shown  in  Figures  24-27,  for  the  signal  amplitude  (A=l). 
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Figure  24:  Probability  of  true  deteetion  vs  probability  of  false  alarm. 

Figure  24  represents  receiver  operating  characteristics  curves  in  the  case  of  our  modified 
(2,  3)  DMC,  for  fixed  value  of  the  noise  powers  a  as  the  decision  threshold  0  increases  from 
zero.  As  the  decision  threshold  0  increases  the  probability  of  true  detection  decreases.  As  the 
decision  threshold  is  symmetric,  positive  increase  of  0  is  also  manifested  in  the  increase  of  the 
negative  threshold  0,  thus  the  probability  of  false  alarm  is  also  increased.  The  probability  of 
false  alarm  curves  reaches  a  maximum  (0.6)  value  as  shown  in  the  figure  above. 


Figure  25:  Probability  of  miss  vs  probability  false  alarm. 


30 


The  decrease  of  the  probability  of  detection  with  increase  of  the  threshold  is  verified  with 
the  increase  of  the  probability  of  miss  as  shown  in  the  Figure  25  by  using  (33.2-3).  For  optimum 
system  performance  we  need  to  have  a  detector  which  minimizes  both  probability  of  miss  and 
false  alarm. 


04  0.5  0.6  0.7  0.8  0.9  1 

Probabilitv'  of  rejection 


Figure  26:  Probability  of  detection  vs  probability  of  rejection. 

A  conservative  detection  is  achieved  by  maximizing  the  probabilities  of  true  detection 
and  rejection.  As  depicted  in  Figure  26,  increasing  the  threshold  0  results  in  a  liberal  system  for 
which  our  detection  performance  decrease  as  more  and  more  false  alarms  are  generated. 
However,  as  the  decision  threshold  0  regresses  towards  the  (2,  2)  DMC  optimum  threshold, 
probability  of  true  detection  is  maximized,  as  the  system  blocks  the  false  alarm  by  maximizing 
the  probability  of  rejection. 


Figure  27:  Probability  of  miss  vs  probability  rejection. 
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For  better  target  detection  and  tracking,  we  need  to  minimize  the  probability  that  we  miss 
a  target  by  maximizing  our  detection.  At  the  same  time  the  false  alarm  that  is  generated  should 
be  minimized  by  maximizing  the  probability  of  declaring  there  is  no  target  in  the  absence  of  a 
valid  target.  As  it  is  depicted  above,  as  the  decision  threshold  theta  exceeds  our  signal  amplitude, 
the  probability  that  we  miss  a  valid  target  increases.  Similarly,  the  detector  generates  a  lot  of 
false  alarms  limiting  the  capability  of  rejecting  a  target  when  it  is  absent. 

Summing  up,  in  Chapter  3  we  first  incorporated  the  (2,  2)  and  the  simple  (2,  3)  DMCs 
optimum  detection  schemes  in  the  study  of  the  modified  (2,  3)  DMC.  We  then  derived  the 
analytical  relationship  that  relates  the  average  probability  of  error  as  a  function  of  the  signal 
amplitude,  detection  threshold  0  and  noise  power  a.  We  studied  the  trade-off  between  the  signal 
amplitude  and  threshold  0  for  better  target  detection  and  tracking.  We  investigated  how  the 
average  probability  of  error  curve  behaves  in  the  case  of  sub/suprathreshold  signaling.  We 
deduced  the  interval  by  which  the  symmetric  threshold  0  needs  to  be  located  relative  to  the 
signal  amplitude  A  for  optimum  system  performance  in  the  presence  of  additive  Gaussian  noise. 
Moreover,  we  classified  the  average  probability  of  error  into  the  probabilities  of  false  alarm,  true 
detection,  miss  and  rejection  to  study  and  investigate  the  effect  of  the  placement  of  0  relative  to 
the  signal  amplitude.  Furthermore,  we  studied  these  four  probabilities  against  each  other  and 
observed  similar  patterns  as  in  the  case  of  the  average  probability  of  error.  These  above 
mentioned  results  and  observations  will  be  utilized  in  Chapter  4  to  investigate  the  SR  effect  in 
the  (2,  3)  DMC. 
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Chapter  4 


Principles  and  System  Model  of  Stochastic  Resonance  (SR) 

SR  is  a  phenomenon  where  the  presence  of  internal  or  external  input  noise  in  non-  linear 
systems  provides  a  better  system  response  to  the  input  signal  than  in  the  absence  of  noise.  SR 
doesn’t  occur  in  linear  systems.  Thus  for  SR  to  occur  three  basic  ingredients  [20]  as  shown  in 
Figure  28,  are  required,  namely: 

•  Sub/superthreshold  input  signal  X  (t) 

•  A  source  of  noise  which  could  be  either  inherent  in  the  systems  or  that  adds  to  the 
coherent  input  iV(t) 

•  Nonlinear  activation  barrier  or  a  form  of  threshold  0 


4.1  System  Model  of  Stochastic  Resonance  (SR) 


X(t) 


Non-linear 
bistable  systems 


>K(t) 


Nit) 

Figure  28:  System  Model  of  SR.  The  noise  variable  iV(t)  is  ~  (0,cr^),  T(t)  is  the  response  of  the 
bistable  system. 

If  the  above  conditions  are  met  then  the  system  undergoes  resonance-  kind  of  behavior 
as  a  function  of  noise  level.  SR  has  been  observed  in  a  large  variety  of  systems,  including  schmit 
trigger  electronic  circuits,  bidirectional  ring  lasers  and  cray  fish  mechanoreceptors 
[3],[9],[11],[13]. 
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The  counterintuitive  effect  relies  in  systems  non  linearity  and  some  parameter  ranges 
being  ‘suboptimaT.  Stochastic  resonance  has  been  observed,  quantified  and  described  a  plethora 
of  physical  and  biological  systems,  including  neurons, [8], [12],  Being  a  topic  of  widespread 
multidisciplinary  interest,  the  definition  of  stochastic  resonance  has  evolved  significantly  over 
the  last  decade  or  so,  leading  to  a  number  of  debates,  misunderstandings  and  controversies. 

Using  (33.1)  the  probability  of  correct  detection  is  plotted  as  a  function  of  the  noise 
power  level  as  shown  in  Figure  31  .  It  is  observed  that  the  probability  of  detection  increases  as 
the  noise  level  increases,  until  the  probability  of  detection  reaches  a  resonance  maximum.This 
increase  in  probability  of  correct  detection  is  not  in  accordance  with  the  intuition  that  increasing 
the  noise  power  level  will  decrease  the  probability  of  correct  detection. 

In  [10]  and  [II],  the  SR  effect  is  observed  on  the  binary-input  binary-output  (2,2)  DMC 
Probability  of  error,  where  minimum  probability  of  error  occurs  at  an  optimal  power  a  of  the 
additive  Gaussian  noise(AGN)  in  relation  to  a  given  threshold  level ,  as  depicted  in  Figure  15. 

4.2  Sub/  Superthreshold  input  signal 

In  this  thesis,  following  Ira  Moskovitz  work  in  [6],  we  analyzed  the  physical 
communication  model  of  the  (2,3)  DMC  for  threshold  based  SR  due  to  additive  noise.  We 
defined  symmetric  decision  threshold  as  shown  in  Figure  18  and  by  conditioning  the  probability 
of  false  alarm  and  miss  to  the  optimum  regions  in  the  (2,  2)  DMC,  the  average  probability  of 
error  as  a  function  of  the  decision  threshold  theta,  noise  power  sigma  and  signal  amplitude  is 
analyzed  for  better  system  performance  in  the  presence  of  additive  guassian  noise. 

In  Chapter  3,  we  investigated  the  average  probabiltiy  of  error  curves  in  the  context  of  the 
modified  (2,  3)  DMC.  We  fixed  the  amplitude  of  the  signal  and  by  varying  the  signal  to  noise 
ratio,  we  examined  the  relationship  between  the  detection  threshold  0  and  the  signal  amplitude. 
However,  to  study  the  effect  of  SR  in  the  modified  (2,  3)  DMC,  we  need  to  examine  the  average 
probability  of  error  or  probability  of  detection  relative  to  the  noise  power  by  fixing  the  signal 
amplitude. 

In  order  to  investigate  the  effect  of  suprathreshold  barrier  (0>A),  we  generated  Figure  29 
using  (33.1).  As  the  detection  0  is  greater  than  the  signal  amplitude,  there  exist  no  noise 
enhanced  signal  detection.  The  detection  performance  of  the  detector  deteriorates  as  the  noise 
power  a  increases.  This  observation  is  consistent  with  the  probability  of  error  curves  that  we 
discussed  in  chapter  3. 
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Figure  29;  probability  of  detection  in  the  (2,  3)  DMC.  The  signal  amplitude  is  1. 

As  the  detection  threshold  0  becomes  greater  than  the  signal  amplitude  in  the  (2,  3)  DMC 
as  shown  in  Figure  29,  there  is  no  noise  enhanced  signal  detection. 
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Figure  30;  Performance  metrics  in  the  (2,  3)  DMC.  The  solid  line  represents  probability  of 
correct  detection,  while  the  dotted  line  represents  probability  of  error. 

In  the  modified  (2,  3)  DMC,  if  the  threshold  0  is  greater  than  the  signal  amplitude,  the 
probability  of  error  increases  as  the  noise  power  a  increases  as  shown  in  Figure  30.  From  this 
observation  and  the  investigations  that  we  did  in  chapter  3,  we  claim,  the  SR  effect  can  only 
occur  in  the  interval  [-0,  0]  for  which  0<  the  signal  amplitude  A. 
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4.3  SR  and  the  (2,  3)  DMC 


Tiny,  low-power,  low-cost  transceiver,  sensing,  and  processing  units  need  to  be  designed 
for  a  successful  deployment  of  underwater  distributed  wireless  sensor  networks.  As  target 
detection  and  tracking  is  essential  on  those  distributed  networks,  we  re-examined  the  (2,  3)  DMC 
in  three  dimensional  view  to  observe  stochastic  resonance  effect.  We  observed  the  phenomenon 
of  SR  on  the  probability  of  signal  detection  over  a  specific  range  of  0  and  noise  variance  a.  In 
Figure  31  using  (33.1)  we  plotted  probability  of  correct  detection  for  different  value  of  the 
threshold  0.  For  a  fixed  threshold  the  performance  of  the  detector  increase  until  it  reaches  its 
peak  value  and  it  starts  to  decrease  as  the  noise  value  increases  beyond  the  optimum  value.  This 
resonant  effect  can  be  easily  observed  in  the  3-D  plot.  As  what  it  is  clearly  shown  in  the  Figure 
32  the  signal  detection  reaches  maximum  in  a  range  which  is  both  dependent  on  the  decision 
threshold  and  the  noise  variance.  As  the  noise  increases  beyond  a  certain  limit,  the  performance 
of  the  detector  deteriorates  and  adding  additional  amount  of  noise  becomes  useless  in  accordance 
with  common  sense.  However,  there  exists  an  optimum  noise  which  gives  better  system 
performance  as  shown  in  Figure  3 1 .  In  the  other  hand,  as  the  decision  threshold  theta  increase  far 
away  from  our  signal  amplitude  as  shown  in  Figures  29-30  ,  the  performance  of  the  modified  (2, 
3)  DMC  deteriorates. 
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In  underwater  distributed  wireless  sensor  networks,  signal  deteetion  and  estimation 
problem  is  on  how  to  maximize  the  detectability  of  a  target  by  minimizing  the  probabilities  of 
false  alarm  and  miss.  As  depicted  in  Figure  32  using  (33.1),  the  increase  of  the  signal  amplitude 
from  A=1  to  1.2  boosts  the  signal  detection.  Thus  by  carefully  selecting  the  systems  parameters, 
in  the  presence  of  additive  white  Gaussian  noise,  one  can  achieve  better  signal  detection  and 
estimation.  As  we  need  to  adhere  to  the  power  constraints  that  we  have  in  case  of  the  UWC,  we 
can  alternatively  select  the  symmetric  detection  threshold  0  to  maximize  our  target  detection. 


Figure  32:  SR  in  (2,  3)  DMC  for  different  value  of  signal  amplitude. 


4.4  Performance  comparison  of  the  (2,  2)  and  (2,  3)  DMCs 

Regardless  of  which  type  of  medium  access  scheme  is  used  for  sensor  networks,  it 
important  for  the  detector  to  detect  the  target  with  less  probability  of  error.  Relationships  (20) 
and  (33.1)  are  used  to  compare  the  performance  of  the  (2,  2)  and  the  (2,  3)  DMCs  when  the 
threshold  0  is  greater  than  the  signal  amplitude.  The  (2,  2)  DMC  exhibits  noise  enhanced 
detection  for  0  >  signal  amplitude  as  shown  in  the  figure  below.  However,  the  performance  of 
the  (2,  3)  DMC  worsens.  But,  for  0<  the  signal  amplitude  the  performance  of  the  modified  (2,  3) 
DMC  is  much  better  than  the  (2,  2)  DMC  as  shown  in  Figure  31.  The  probability  of  detection  (2, 
3)  DMC  increases  when  the  noise  power  a  increases  until  it  reaches  its  optimum  value.  Further 
addition  of  noise  deteriorates  the  performance  of  the  detectors  and  probability  of  detection 
decreases. 
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Figure  33:  Probabilities  of  deteetion  in  the  (2,  2)  and  (2,  3)  DMCs.  The  solid  and  dotted  line 
represent  probabilities  of  detection  in  the  (2  3)  and  (2,  2)  DMCs  respectively. 

As  the  detection  threshold  0  is  greater  than  our  signal  amplitude,  we  observe  noise 
enhanced  better  signal  detection  with  less  probability  of  error  in  the  case  of  the  (2,  2)  DMC  as 
shown  in  Figures  33-34. 


Figure  34:  Probability  error  in  the  (2,  2)  and  (2,  3)  DMCs.  The  solid  and  dotted  line  represent 
probabilities  of  error  in  the  (2  3)  and  the  (2,  2)  DMCs  respectively. 


38 


Another  important  comparison  can  be  drawn  between  the  two  (2,  3)  DMCs;  the  average 
probability  of  error  in  the  simple  (2,  3)  DMC,  can  be  calculated  as  shown  in  the  Figure  35  as 
follows. 


Figure  35:  A  simple  (2,  3)  DMC  detection  scheme.  The  errors  associated  with  bit  0  and  1  are 
represented  by  the  areas  Dq  and  respectively. 

The  error  associated  with  bit  0  is  the  area  under  the  pdf,  (y  |5’o)  >  —6  .  Thus 

/— 0  —  i4o\ 

Similarly  the  error  associated  with  bit  1  is  the  areas  under  the  pdf,  /(y|si)  <  6. 


Using  (19),  thus  the  average  probability  of  error  is  given  by 

P(e|e)  =  Po  X  +p^x(l-Q  (^)) 

The  average  probability  of  detection  in  the  simple  (2,  3)  DMC  can  be  obtained  as  follows: 


(35) 


P(d|e)  =  1  -  [po  X  Q{^)  +p^x(l-Q  (^)) 


(35.1) 


The  average  probabilities  of  error  in  the  (2,  2),  the  simple  (2,  3)  and  our  modified  (2,  3) 
DMCs  are  modeled  as  Q  functions  as  can  be  calculated  as  in  (20),  (33)  and  (35).  We  have  also 
discussed  the  intervals  by  which  the  SR  effect  is  observed.  The  modified  (2,  3)  exhibits  SR  for  a 
detection  threshold  0  less  than  the  signal  amplitude,  while  the  other  two  DMCs  for  0  greater 
than  the  signal  amplitude  [5].  We  generated  a  three  dimensional  plot  to  investigate  and  compare 
the  performances  of  the  three  DMCs  as  shown  in  Figure  36. 


39 


o  1 

o 

0.8 

D 

•O 

u-  0.6 
o 

^0.4 

0.2 

&  0 


Comparision 


Figure  36;  DMCs  performance  Comparison.  Using  (20.1),  (35.1)  and  (33.1). 

An  ideal  sensor  network  is  designed  to  detect  a  target  with  negligible  probability  of  false 
alarm  and  miss.  However,  we  cannot  reach  that  peak  due  to  channel  imperfections.  Thus  the 
designer  task  would  be  to  compare  different  channel  models  and  come  up  with  a  model  that 
minimizes  the  probability  of  error  relatives  to  the  other  feasible  channel  models.  In  both  (2,  3) 
DMCs  the  probability  of  detection  deteriorates  so  fast  as  the  symmetric  decision  threshold  theta 
increases  far  away  from  the  signal  amplitude.  However,  the  (2,  2)  DMC  exhibits  slow  decay  in 
the  probability  of  detection  as  the  single  decision  threshold  theta  increases  far  away  from  the 
signal  amplitude.  This  slow  decrease  in  the  probability  of  detection  is  from  the  fact  that, 
increasing  the  threshold  increases  the  probability  of  missing  a  target.  But,  the  reduction  in 
probability  of  false  alarm  helps  to  compensate  the  increase  in  the  probability  of  miss,  which 
ultimately  balances  the  average  probability  of  error. 

To  the  best  of  our  knowledge,  although  many  channel  models  are  proposed  [3]-[12],  we 
showed  our  modified  (2,  3)  DMC  has  better  performance  in  detecting  a  target  by  minimizing  the 
probability  of  error  which  is  inherent  to  any  communication  system.  The  results  of  this  thesis 
clearly  indicate  how  one  could  compare  the  performance  of  the  (2,  2)  and  (2,  3)  DMCs.  We  also 
investigated  how  one  could  improve  the  probability  of  correct  detection  in  the  threshold  based 
communication  systems,  where  local,  finite,  and  relatively  finite  energy  sources  are  deployed  to 
monitor  a  geographical  area. 
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4.4.1  Stochastic  Resonance  and  Forbidden  Interval  Theorem  in  the  case  of  the 
BSC 


In  [14],  the  Forbidden  Interval  Theorems  (FIT)  in  stoehastic  resonance  is  introduced.  The 
FIT  gives  the  necessary  and  the  sufficient  condition  for  stochastic  resonance  to  occur.  In  general, 
the  forbidden  interval  is  the  region  or  interval  in  which  there  is  no  stochastic  resonance.  A 
similar  approach  is  reported  in  [15],  where  the  Algebraic  Information  Theory  (AIT)  was  used  for 
the  (2,  2)  DMC  to  determine  the  forbidden  interval.  Moreover,  in  [5]  the  forbidden  interval 
theorem  is  expanded  into  (2,  3)  DMC  while  enhancing  the  channel  capacity  by  addition  of  noise. 

As  our  performance  measure  is  the  probability  of  error  we  studied  the  noise  enhanced 
signal  detection  and  identify  the  intervals  by  which  the  effect  of  stochastic  resonance  is 
observed.  We  first  analyzeed  the  (2,  2)  DMC  to  lay  down  a  foundation  for  the  later  anaysis  of 
the  (2,  3)  DMC. 


Differentiating  (20)  with  respect  to  the  noise  standard  deviation  a,  we  get 


dp(e\9) 


a  r°°  1  \  2 

d  Iff-Ap  ^  *e 


3  1 


=  Po 


da  da 

Simplifying  (36.1)  we  obtain 


Pi 


da 


O^Pox(o-e  ^  X  -  +  p^x(o-e  ^  x 

Po  X  (o  -  ^  X  (^)  =  Pi  X  1^0  - 
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Rearranging  we  get 


Using  logarithm  in  both  sides,we  obtain 


(36.1) 


(36.2) 

(36.3) 


(36.4) 
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(36.5) 
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Thus  the  optimum  noise  is  as  a  function  of  signal  amplitude  and  decision  threshold  0  and  the 
prior  probabilities. 

However,  in  order  (36.5)  to  be  defined  then  the  threshold  0  should  be  greater  than  the  signal 
amplitude  and  Aq.  Thus  SR  occurs  in  the  interval  Aq<A^<Q 

For  the  (2,  3)  DMC  shown  in  the  Figure  36,  in  order  to  study  the  forbidden  interval  theorem  we 
followed  the  same  procedure  as  we  did  with  the  (2,  2)  DMC. 

Differentiating  (35)  with  respect  to  the  noise  standard  deviation  a  and  rearranging,  we  get 


Solving  for  the  optimum  noise,  we  get 


20(y4]^  +Aq^+Aq^—A^^ 

^  Pi(e-^i)  J 

The  optimum  value  of  the  noise  obtained  in  (37.2)  is  defined  if  and  only  if 

-  0  <i4o<i4i<  0 

The  above  interval  for  which  the  SR  phenomena  observed  is  consistent  with  the  intervals  by 
which  the  channel  capacity  exhibits  SR  in  [5]. 


(37.2) 


4.4.2The  optimum  noise  for  minimum  probability  of  error  in  the  (2,  2)  DMC 


The  noise  enhancement  for  signal  detection  in  the  case  of  (2,  2)  DMC  depends  on  the 
location  of  the  detection  threshold  relative  to  the  signal  amplitude.  As  it  is  deduced  in  (36.5)  the 
threshold  theta  should  be  greater  than  the  signal  amplitude  in  order  the  system  to  exploit  the  SR 
effect.  In  Figure  37  we  observed  the  optimum  noise  to  be  zero  a  long  as  the  threshold  0  is  less 
than  the  signal  amplitude  (A=l).  SR  cannot  happen  in  this  interval.  However,  for  the  detection 
threshold  0  greater  than  the  signal  amplitude,  as  shown  in  the  figure  below,  there  is  an  optimum 
value  of  the  noise  that  contributes  to  better  signal  detection. 
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Figure  37;  Optimal  noise  power  a  vs  threshold  0.  The  signal  amplitude  is  1. 


4.4.3  Stochastic  Resonance  and  the  modified  (2,  3)  DMC 


Following  our  work  in  the  ease  of  the  (2,  2)  DMC  and  the  (2,  3)  DMC,  we  revisited  (33) 
Rewriting  and  differentiating  (33)  with  respect  to  noise  standard  deviation  a,  we  get 

X  )  X  (A)  _  X  ()A)  .  ,  0  ,  ^  X 

(V»  =©^)  X  «  (^)  -  A  X  X  5  (^A)  +  A  X  X  5  (0  =  0  ,38. 


Collecting  like  terms,  we  get 


(^) 


X  e 


X  1 


-^e)) 


=  ^  X  e 


X  1 


1) 


(38.2) 


To  deduce  the  interval  by  which  the  noise  enhancement  effect  exists  we  find  a  relationship  in 
terms  of  sigma  as  follows: 


a  = 


(38.3) 


In  order  (38.3)  to  be  defined  then  i4  >  6,  i.e.  the  signal  need  to  superthreshold  . 

In  the  special  case  when  the  threshold  theta,0=O,  the  (2,  3)  DMC  becomes  (2,  2)  DMC,  and 
we  know  this  is  optimum  regardless  of  the  noise.. 
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Chapter  5 

Conclusions  and  future  work 


In  this  thesis,  we  analyzed  the  (2,  3)  DMC  and  formulated  the  average  probability  of 
error  as  a  funetion  of  signal  amplitude,  threshold  theta  and  noise  power  sigma.  We  studied  the 
trade-off  between  the  detection  threshold  and  signal  amplitude  for  the  average  probability  of 
error.  We  also  studied  how  the  receiver  operating  characteristics  curves  in  case  of  the  (2,  3) 
DMC  shifts  with  the  detection  threshold  theta  and  noise  power  sigma. 

We  found  the  signal  has  to  be  subthreshold  in  the  case  with  BSC  and  the  simple  (2,  3) 
DMC.  However,  in  our  proposed  (2,  3)  DMC  model  the  signal  has  to  be  superthreshold. 
Moreover,  we  showed  our  (2,  3)  DMC  performs  better  than  the  simple  (2,  3)  DMC 

In  addition,  we  studied  the  threshold  based  stochastic  resonance  behavior  of  binary-input 
ternary-output  DMC.  The  physical  communication  channel  model  is  used  to  analyze  the  average 
probability  of  error  of  the  BSC/SE  in  the  context  of  the  stochastic  resonance  phenomenon  in 
order  to  investigate  SR  effect  in  the  modified  (2,  3)  DMC.  We  observed  the  existence  of  an 
optimum  noise  in  case  of  nonlinear  systems  which  is  a  function  of  our  detection  threshold  0. 

To  the  best  of  our  knowledge,  although  many  channel  models  are  proposed  [3]-[12],  we 
showed  our  modified  (2,3)  DMC  has  better  performance  in  detecting  a  target  by  minimizing  the 
probability  of  error  which  is  inherent  to  any  communication  systems  The  results  of  this  thesis 
clearly  indicate  how  one  could  compare  the  performance  of  the  (2,2)  and  (2,3)  DMCs.  The  result 
of  our  work  can  be  used  in  threshold  based  communication  systems,  where  local,  finite,  and 
relatively  finite  energy  sources  are  deployed  to  monitor  a  geographical  area. 

As  our  proposed  model  deals  with  a  static  threshold  0,  the  future  work  can  be  expanded 
with  dynamic  threshold  employing  the  same  model.  Especially  when  the  precise  location  of  a 
target  is  of  high  importance  in  the  case  of  distributed  wireless  sensor  networks,  it  is  necessary 
that  we  dynamically  set  the  threshold  0  comparing  the  different  reading  from  the  distributed 
sensors.  The  rate  of  true  detection  to  the  total  detection  can  be  used  to  dynamically  set  our 
threshold  0.  Eurthermore  it  would  be  more  interesting  to  investigate  the  effect  of  noise 
simultaneously  using  both  information  theory  and  communication  channel  parameters.  Does  the 
optimum  value  of  noise  that  boosts  our  signal  detection  also  maximize  the  channel  capacity  and 
mutual  information  of  a  channel?  Thus,  one  can  optimize  the  system  to  efficiently  use 
communication  bandwidth  and  energy  which  are  both  scarce  in  the  case  of  underwater 
distributed  wireless  sensor  networks. 
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